A Methodology of Error Detection: Improving Speech Recognition in Radiology

Automated speech recognition (ASR) in radiology report dictation demands highly accurate and robust recognition software. Despite vendor claims, current implementations are suboptimal, leading to poor accuracy, and time and money wasted on proofreading. Thus, other methods must be considered for increasing the reliability and performance of ASR before it is a viable alternative to human transcription. One such method is post-ASR error detection, used to recover from the inaccuracy of speech recognition. This thesis proposes that detecting and highlighting errors, or areas of low confidence, in a machine-transcribed report allows the radiologist to proofread more efficiently. This, in turn, restores the benefits of ASR in radiology, including efficient report handling and resource utilization. To this end, an objective classification of error-detection methods for ASR is established. Under this classification, a new theory of error detection in ASR is derived from the hybrid application of multiple error-detection heuristics. This theory is contingent upon the type of recognition errors and the complementary coverage of the heuristics. Inspired by these principles, a hybrid error-detection application is developed as proof of concept. The algorithm relies on four separate artificial-intelligence heuristics together covering semantic, syntactic, and structural error types, and developed with the help of 2700 anonymised reports obtained from a local radiology clinic. Two heuristics involve statistical modeling: pointwise mutual information and co-occurrence analysis. The remaining two are non-statistical techniques: a property-based, constraint-handling-rules grammar, and a conceptual distance metric relying on the ontological knowledge in the Unified Medical Language System. When the hybrid algorithm is applied to thirty real-world radiology reports, the results are encouraging: up to a 24% increase in the recall performance and an 8% increase in the precision performance over the best single technique. In addition, the resulting algorithm is efficient and modular.